{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "# Reference: https://jupyterbook.org/interactive/hiding.html\n",
    "# Use {hide, remove}-{input, output, cell} tags to hiding content\n",
    "\n",
    "import sys\n",
    "import os\n",
    "sys.path.append(os.path.abspath('../../..'))\n",
    "from textbook_utils import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "trees = pd.read_csv('data/black_spruce.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "trees['status_0_1'] = 0 + (trees['status'] == 'fallen')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "splits = [4, 5, 6, 7, 8, 9, 10, 12, 14, 17, 20, 30]\n",
    "pr_split = np.zeros(len(splits) - 1)\n",
    "ct_split = np.zeros(len(splits) - 1)\n",
    "\n",
    "for i in range(0, len(splits) - 1):\n",
    "    sub_trees = trees[\"status_0_1\"][\n",
    "        (\n",
    "            (trees[\"diameter\"] > splits[i])\n",
    "            & (trees[\"diameter\"] <= splits[i + 1])\n",
    "        )\n",
    "    ]\n",
    "    pr_split[i] = np.mean(sub_trees)\n",
    "    ct_split[i] = len(sub_trees)\n",
    "\n",
    "tree_bins = pd.DataFrame(\n",
    "    {\"proportion\": pr_split, \"count\": ct_split, \"diameter\": splits[1:]}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Loss Function for the Logistic Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logistic model gives us probabilities (or empirical proportions), so we write our loss function as $\\ell(p, y) $, where $p$ is between 0 and 1. The response takes on one of two values because our outcome feature is a binary classification. Thus, any loss function reduces to: \n",
    "\n",
    "$$\n",
    "\\begin{aligned} \n",
    "{\\ell}(p, y) = \n",
    " \\begin{cases}\n",
    "    \\ell(p, 0) & \\text{if $y$ is 0} \\\\    \n",
    "    \\ell(p, 1) & \\text{if $y$ is 1}\n",
    "    \\end{cases}\n",
    "\\end{aligned} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, using 0 and 1 to represent the categories has an advantage because we can conveniently write the loss as:\n",
    "\n",
    "$$ \n",
    "\\ell(p, y) =  ~ y \\ell(p, y) + (1-y) \\ell(p, 1-y)            \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We encourage you to confirm this equivalence by considering the two cases $y=1$ and $y=0$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logistic model pairs well with *log loss*:\n",
    "\n",
    "$$\n",
    "\\begin{aligned} \n",
    "{\\ell}(p, y) = & \n",
    " \\begin{cases}\n",
    "    -\\log(p) & \\text{if $y$ is 1} \\\\    \n",
    "    -\\log(1-p) & \\text{if $y$ is 0}\n",
    "    \\end{cases}\n",
    "    \\\\\n",
    "    = & -y \\log(p) -(1-y) \\log(1-p)   \n",
    "\\end{aligned} \n",
    "$$ \n",
    "\n",
    "Note that the log loss is not defined at 0 and 1 because $-\\log (p)$ tends to $\\infty$ as $p$ approaches 0, and similarly for $-\\log (1-p)$ as $ p $ tends to 1. We need to be careful to avoid the end points in our minimization. We can see this in the following plot of the two forms of the loss function: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "line": {
          "width": 4
         },
         "name": "y = 1",
         "type": "scatter",
         "x": [
          0.001,
          0.014486486486486486,
          0.027972972972972975,
          0.04145945945945946,
          0.05494594594594595,
          0.06843243243243244,
          0.08191891891891892,
          0.09540540540540542,
          0.1088918918918919,
          0.12237837837837838,
          0.13586486486486488,
          0.14935135135135136,
          0.16283783783783784,
          0.17632432432432432,
          0.18981081081081083,
          0.2032972972972973,
          0.2167837837837838,
          0.23027027027027028,
          0.24375675675675676,
          0.25724324324324327,
          0.27072972972972975,
          0.28421621621621623,
          0.2977027027027027,
          0.3111891891891892,
          0.3246756756756757,
          0.33816216216216216,
          0.35164864864864864,
          0.3651351351351352,
          0.37862162162162166,
          0.39210810810810814,
          0.4055945945945946,
          0.4190810810810811,
          0.4325675675675676,
          0.44605405405405407,
          0.45954054054054055,
          0.47302702702702704,
          0.4865135135135135,
          0.5,
          0.5134864864864865,
          0.526972972972973,
          0.5404594594594595,
          0.5539459459459459,
          0.5674324324324325,
          0.580918918918919,
          0.5944054054054054,
          0.607891891891892,
          0.6213783783783784,
          0.6348648648648649,
          0.6483513513513514,
          0.6618378378378379,
          0.6753243243243243,
          0.6888108108108109,
          0.7022972972972973,
          0.7157837837837838,
          0.7292702702702704,
          0.7427567567567568,
          0.7562432432432433,
          0.7697297297297298,
          0.7832162162162163,
          0.7967027027027027,
          0.8101891891891893,
          0.8236756756756757,
          0.8371621621621622,
          0.8506486486486486,
          0.8641351351351352,
          0.8776216216216217,
          0.8911081081081081,
          0.9045945945945947,
          0.9180810810810811,
          0.9315675675675676,
          0.9450540540540541,
          0.9585405405405406,
          0.972027027027027,
          0.9855135135135136,
          0.999
         ],
         "y": [
          6.907755278982137,
          4.23453903055556,
          3.576516485926892,
          3.1830392096991598,
          2.9014053779980147,
          2.6819084083622946,
          2.5020253145685585,
          2.3496200416990187,
          2.2173997064350144,
          2.1006375717433428,
          1.9960945279337863,
          1.9014536861429996,
          1.8150004332675058,
          1.735430227885457,
          1.6617274354673408,
          1.5930858525960798,
          1.5288548102594368,
          1.468501571803,
          1.4115844494606533,
          1.3577331699453055,
          1.306634263014597,
          1.2580200057765896,
          1.211659932389918,
          1.1673542263906065,
          1.1249285159049862,
          1.0842297288412692,
          1.0451227591228858,
          1.0074877606721167,
          0.9712179324302475,
          0.9362176912235685,
          0.9024011537303718,
          0.8696908668474571,
          0.838016739222398,
          0.8073151368752737,
          0.7775281135688756,
          0.7486027525305851,
          0.7204906007356568,
          0.6931471805599453,
          0.6665315664418214,
          0.6406060164371447,
          0.6153356503413585,
          0.5906881674898461,
          0.5666335985080677,
          0.5431440862252627,
          0.5201936917346651,
          0.4977582222142691,
          0.47581507764242503,
          0.4543431139734029,
          0.43332252069643934,
          0.412734711001113,
          0.3925622230229113,
          0.372788630854137,
          0.35339846418381926,
          0.33437713558162474,
          0.31571087456950153,
          0.2973866677346418,
          0.27939220423138816,
          0.2617158261004494,
          0.2443464829032986,
          0.22727369022965824,
          0.21048749168792086,
          0.19397842403346122,
          0.17773748512901824,
          0.1617561044655518,
          0.1460261160018817,
          0.13053973310762773,
          0.11528952541696916,
          0.10026839742098911,
          0.08546956864421225,
          0.07088655526670398,
          0.056513153067055115,
          0.04234342157393693,
          0.028371669324904897,
          0.01459244014089594,
          0.0010005003335835344
         ]
        },
        {
         "line": {
          "dash": "dot",
          "width": 4
         },
         "name": "y = 0",
         "type": "scatter",
         "x": [
          0.001,
          0.014486486486486486,
          0.027972972972972975,
          0.04145945945945946,
          0.05494594594594595,
          0.06843243243243244,
          0.08191891891891892,
          0.09540540540540542,
          0.1088918918918919,
          0.12237837837837838,
          0.13586486486486488,
          0.14935135135135136,
          0.16283783783783784,
          0.17632432432432432,
          0.18981081081081083,
          0.2032972972972973,
          0.2167837837837838,
          0.23027027027027028,
          0.24375675675675676,
          0.25724324324324327,
          0.27072972972972975,
          0.28421621621621623,
          0.2977027027027027,
          0.3111891891891892,
          0.3246756756756757,
          0.33816216216216216,
          0.35164864864864864,
          0.3651351351351352,
          0.37862162162162166,
          0.39210810810810814,
          0.4055945945945946,
          0.4190810810810811,
          0.4325675675675676,
          0.44605405405405407,
          0.45954054054054055,
          0.47302702702702704,
          0.4865135135135135,
          0.5,
          0.5134864864864865,
          0.526972972972973,
          0.5404594594594595,
          0.5539459459459459,
          0.5674324324324325,
          0.580918918918919,
          0.5944054054054054,
          0.607891891891892,
          0.6213783783783784,
          0.6348648648648649,
          0.6483513513513514,
          0.6618378378378379,
          0.6753243243243243,
          0.6888108108108109,
          0.7022972972972973,
          0.7157837837837838,
          0.7292702702702704,
          0.7427567567567568,
          0.7562432432432433,
          0.7697297297297298,
          0.7832162162162163,
          0.7967027027027027,
          0.8101891891891893,
          0.8236756756756757,
          0.8371621621621622,
          0.8506486486486486,
          0.8641351351351352,
          0.8776216216216217,
          0.8911081081081081,
          0.9045945945945947,
          0.9180810810810811,
          0.9315675675675676,
          0.9450540540540541,
          0.9585405405405406,
          0.972027027027027,
          0.9855135135135136,
          0.999
         ],
         "y": [
          0.0010005003335835344,
          0.014592440140896053,
          0.028371669324904897,
          0.04234342157393705,
          0.056513153067055115,
          0.07088655526670411,
          0.08546956864421225,
          0.10026839742098922,
          0.11528952541696916,
          0.13053973310762787,
          0.1460261160018817,
          0.1617561044655518,
          0.17773748512901838,
          0.19397842403346122,
          0.21048749168792097,
          0.22727369022965824,
          0.24434648290329875,
          0.2617158261004494,
          0.27939220423138833,
          0.2973866677346418,
          0.3157108745695017,
          0.3343771355816249,
          0.35339846418381926,
          0.372788630854137,
          0.3925622230229113,
          0.41273471100111314,
          0.43332252069643934,
          0.45434311397340305,
          0.47581507764242503,
          0.49775822221426924,
          0.5201936917346653,
          0.5431440862252629,
          0.5666335985080677,
          0.5906881674898461,
          0.6153356503413587,
          0.6406060164371447,
          0.6665315664418214,
          0.6931471805599453,
          0.720490600735657,
          0.7486027525305851,
          0.7775281135688757,
          0.8073151368752737,
          0.8380167392223982,
          0.8696908668474573,
          0.9024011537303719,
          0.9362176912235688,
          0.9712179324302476,
          1.007487760672117,
          1.0451227591228858,
          1.0842297288412692,
          1.1249285159049862,
          1.1673542263906067,
          1.211659932389918,
          1.2580200057765898,
          1.3066342630145975,
          1.3577331699453057,
          1.4115844494606535,
          1.4685015718030001,
          1.5288548102594373,
          1.59308585259608,
          1.6617274354673413,
          1.735430227885457,
          1.815000433267506,
          1.9014536861429996,
          1.9960945279337867,
          2.1006375717433436,
          2.217399706435015,
          2.34962004169902,
          2.502025314568559,
          2.681908408362296,
          2.901405377998015,
          3.1830392096991615,
          3.5765164859268923,
          4.234539030555563,
          6.907755278982136
         ]
        }
       ],
       "layout": {
        "height": 250,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autosize": true,
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "height": 250,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 10,
           "l": 10,
           "r": 10,
           "t": 10
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.5,
           "xanchor": "center"
          },
          "width": 350,
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": true,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": true,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "width": 450,
        "xaxis": {
         "range": [
          0,
          1
         ],
         "type": "linear"
        },
        "yaxis": {
         "autorange": true,
         "range": [
          -0.3827080984802251,
          7.2914638777959455
         ],
         "type": "linear"
        }
       }
      },
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAy4AAAD6CAYAAABH9BMPAAAAAXNSR0IArs4c6QAAIABJREFUeF7snQd0VNX2xj9COgkkgdB7F6WoIMVGEUWKCCoIVhQVuz6Vvz59ihV5NuyAAj4sYMOGYgEBEZQiIE2Q3kFqeif/te9k5s6ElDszd+a276w1K7m5556z97dPkvnNOWefKsXFxcVgoQJUgApQASpABagAFaACVIAKmFiBKgQXE0eHplEBKkAFqAAVoAJUgApQASqgKEBw4UCgAlSAClABKkAFqAAVoAJUwPQKEFxMHyIaSAWoABWgAlSAClABKkAFqADBhWOAClABKkAFqAAVoAJUgApQAdMrQHAxfYhoIBWgAlSAClABKkAFqAAVoAIEF44BKkAFqAAVoAJUgApQASpABUyvAMHF9CGigVSAClABKkAFqAAVoAJUgAoQXDgGqAAVoAJUgApQASpABagAFTC9AgQX04eIBlIBKkAFqAAVoAJUgApQASoQcnBp1aoVtmzZQqWpQKUKHD58GDVr1kRERESldVnBuQrk5eUhJycHSUlJzhWBnmtS4MSJE4iLi0NMTIym+qzkTAVOnjyJo0ePIjU11ZkC0GsqYCEFCC4WCpbdTSW42D3C+vhHcNFHRye0QnBxQpSD95HgEryGbIEKhEsBgku4lGY/lSpAcKlUIlYAQHDhMNCqAMFFq1LOrkdwcXb86b21FCC4WCtetraW4GLr8OrmHMFFNylt3xDBxfYh1sVBgosuMrIRKhAWBQguYZGZnWhRgOCiRSXWIbhwDGhVgOCiVSln1yO4ODv+9N5aChBcrBUvW1tLcLF1eHVzjuCim5S2b4jgYvsQ6+IgwUUXGdkIFQiLAgSXsMjMTrQoQHDRohLrEFw4BrQqQHDRqpSz6xFcnB1/em8tBQgu1oqXra0luNg6vLo5R3DRTUrbN0RwsX2IdXGQ4KKLjGyECoRFAYJLWGRmJ1oUILhoUYl1CC4cA1oVILhoVcrZ9Qguzo4/vbeWAgQXa8XL1tYSXGwdXt2cI7joJqXtGyK42D7EujhIcNFFRjZCBfxSoLi4GEVFRYiMjPTrubCAy4gXv8DavWlIqRaNaTd28ctAVnaOAgQX58Q6GE8JLsGo56xnCS7Oineg3hJcAlWOz1GBwBX46quv8OKLL2Lx4sV+NRIWcCm4YqJiVKPkOCz+v95+GcjKzlGA4OKcWAfjKcElGPWc9SzBxVnxDtRbgkugyjnoua/vBqrVBup3BE67zEGO6+/qrl27MGrUKOzZswd169Y1N7hEVAG2PNsfVeUbFipQSgGCC4eEFgUILlpUYh1RgODCcaBFAYKLFpUcXOdkEfB0LaD4pEuEcWm6iHEiuwDTl+zQpS1/GkmKj8aoc5ue8sh9992Hrl27YsSIEco9Wco1ZswYDBkyBP369fOniwrrFhYW4siRI5g3bx4mT55sbnART2TGRWZeWKhAaQUILhwTWhQguGhRiXUILhwDWhUguGhVyqH1ju8AXu2kOq8TuOw8mo2eLywIu6hNasZj0UO9Tul32rRpmD59OhYuXIiqVatixYoVGDlyJJYtW4aUlJRT6i9duhRvv/12mfafc845uPvuuyv07dtvv8Xzzz9vfnD56JZu6NGiZtgDxQ7NrwDBxfwxMoOFBBczRMEaNnDGxRpxMtpKgovRETB5/9sXADMutz24HDt2TJlxmTp1Ki644AI8+OCDiIqKwvjx48sM0MGDB7Fhw4Yy76WmpqJDhw72AJcJV3TA8C6NTD5KaZ4RChBcjFDden0SXKwXM6MsJrgYpby1+iW4WCteYbc28x9gy4/AkS3A0a3A1R/qYoLZZlzEqbFjxyIzMxNPPfUUunfvjtmzZ6N9+/Zl+iugs3v37jLvVa9eHc2bN7cHuNzVqyUevKSNLkFnI/ZSgOBir3iGyhuCS6iUtV+7BBf7xTQUHhFcQqEq26xMAbPtcRF7V61aheHDh+Oaa67B2rVrFXApryxYsADPPfdcmbd79uyJRx991B7gclnH+nhtxJmVxZP3HagAwcWBQQ/AZYJLAKI59BGCi0MD76fbBBc/BWN1WyswcOBAbN68WUlVPHjwYN19lU3/BQUFmDt3rtLH/PnzlT018tJSwpoOWQw6s1ESvrjzXC22sY7DFCC4OCzgAbpLcAlQOAc+RnBxYNADcJngEoBofMS2CkyZMgVvvfWWsik/JiZGdz+3bNmC/v37+7Qr+2luu+02TX2FHVxqJURj5WN9NRnHSs5SgODirHgH6i3BJVDlnPccwcV5MQ/EY4JLIKo56JnPbwHikoDkJkByM6DtAFs7L+mQZZO+pEc2Ywk7uIgIm56+FLFREWbUgzYZqADBxUDxLdQ1wcVCwTLYVIKLwQGwSPcEF4sEyggzC3OBZ+r49qxTOmQj3Kmsz3Xr1mHo0KFYtGgR6tevX1l1Q+6HBVzqjp6MPcdzPA7+eP8FaF0n0RCH2al5FSC4mDc2ZrKM4GKmaJjbFoKLueNjFusILmaJhAntOLwZePMc1bCkJsB9a01oqD4mbdy4Efv27UPfvuZdGRUWcOky9n0s3XbUo+rUG7qgz2m19VGZrdhGAYKLbUIZUkcILiGV11aNE1xsFc6QOUNwCZm01m9Y0iB/eJXqR7MLgBu+sb5fFvYgLOAydPzn+HjlHo9MTww6HaPObWph2Wh6KBQguIRCVfu1SXCxX0xD5RHBJVTK2qtdgou94qmrNxkHge0LgRM7geO7gdptgR736NoFG/NPgbCAy71T5uLFHzZ7LLvp3GZ4fFA7/yxlbdsrQHCxfYh1cZDgoouMjmiE4OKIMAftJMElaAnZABUImwJhAZcXP12Ee2et9jjVt10dvHN957A5yY6soQDBxRpxMtpKgovREbBO/wQX68TKSEsJLkaqz76pgH8KhAVcPp6/AkPfWuKxrE3dRPxw3wX+WcratleA4GL7EOviIMFFFxkd0QjBxRFhDtpJgkvQErIBKhA2BcICLktXr0eXZ+Z5nIqJjMDmZy4Nm5PsyBoKEFysESejrSS4GB0B6/RPcLFOrIy0lOBipPom7/vj64DEukBSY9ernf4nyZtcAdOZFxZwkVMy2zw2F3mFJz0CrPpPX6RUizadIDTIOAUILsZpb6WeCS5WipaxthJcjNXfKr0TXKwSqTDbmX0U+G9ztdPIGOCxf8JshH27y8jIQEFBAVJSUvxyMiBwyczMhHRYp04dRERUfJBkq1atIODS95VF2HIo02Pcl3eei06NkvwylpXtrQDBxd7x1cs7goteStq/HYKL/WOsh4cEFz1UtGEbe1cC7/ZRHUttC9y5zIaOhtelrKwsPPDAA5g/f77ScadOnfDOO+8gKUkbE/gFLgsWLMBzzz2HnTt3Kp3NmTMHbdq0qdBjN7jc9N4K/LxJJdXXR5yJQR3NeSpneEPI3twKEFw4FrQoQHDRohLriAIEF44DLQoQXLSo5MA66z4BPr9Fdbx1P2Dkxw4UQl+XJ0+ejI8//hgzZ85EfHw8Ro8ejYsvvhg333yzpo40g8vPP/+M2267TXkNHToUycnJiI2NRVxcnCZwGff1Bry31AU8Usb2a4s7erbQZCQrOUMBgosz4hyslwSXYBV0zvMEF+fEOhhPCS7BqGfzZw+tBw6uAw6sBWo2B7p4gUywruccB36fpLYSnwx0HaNeh/p+Kfvvu+8+dO3aFSNGjFDuFBcXY8yYMRgyZAj69esXrLee5wcPHoz+/fsrPCFl7ty5CsTMmDFDUx+awEWMHzRoENq2bYsXX3xRU8PuSu4Zl6m/7sDTczZ6nr36nEZ4fmgHv9piZXsrQHCxd3z18o7gopeS9m+H4GL/GOvhIcFFDxXZht8KHNsOvHam+lhKc+Ae9egQhPp+KYOnTZuG6dOnY+HChahatSpWrFiBkSNHYtmyZWXuQ1m6dCnefvvtMt0+55xzcPfdd5d5T5aGjR8/Hpde6krStWHDBkyYMEFfcDl27JhCYX369FE20mRnZ6NHjx645ZZblFmXioobXH7ceAi3zljpqXpuy1r4cHRXv+PMB+yrAMHFvrHV0zOCi55q2rstgou946uXdwQXvZRkO34pEGowqaz9Usa63+tPnToVF1xwAR588EFERUUpkFFWOXjwoAIdZZXU1FR06HDq5IRMhLRu3RpTpkxBr169lEe3bt2Kp556Sl9w2bhxI2RqZ/jw4TjvvPOQnp6u7HUZOHAgnnnmGY/NN9100yn2L168GL/99hv+/icbI2es89xvmBSLL0d39CvGrGxvBYqKihTKZ6ECFSkgf/jkjQbHiq9K8g9GPlhiURWQvymSQKZKlSqUhQpUqAD//3CAaFWgVq1aWqtWXK8ysAj1/TKsGzt2LCQBl4BE9+7dMXv2bLRv375MPwR0du/eXea96tWro3lzr4xsXrVkxuX555/3LD8LyYyLG1y8p4s+/fRTPPvss1i9erXnn0JZDsgszaZNm5RUyO2e+NFjekQVYMsz+q2Z02cUsRUjFZBfAskqUVmmOiNtZN/GK5Cfnw+ZdUlMTDTeGBNZIG/OBepYVAUk+2VMTAyio5l6n+OifAXkgxCZnfM3LSs1tbcCVV9ohuLE+sr5LcXy6jdBcVi3D81CvYelsvbLCN+qVauUSYprrrkGa9euVcClvOJO2FXW/Z49e+LRRx8t89Gw7HFJS0tD586dIbAipCRFNtI8/vjj2Lx5c4VvNN1LxeSZM5/6Ecez1U8Ef/2/3miYXPHmfnv/WtA7bwW4VIzjQYsCXCqmRSXWEQW4VIzjQIsCXCqmRSWH1Sk925FQB3jwb0eIIKup5L297GkXyNC7eGcVq1atmpJNLCRZxaRh+TTv9ddfx9GjRyHZB+rXr4833nijQp+8weWyN37F2r1pnvozb+2G7s1r6q0J27OoAgQXiwYuzGYTXMIsuIW7I7hYOHhhNJ3gEkaxrdLVtp+B94eo1jY6B7j5J6tYH5Sdsv/krbfeUjbly4y13kWWot1///1KEgApshRN9tVItmItRVNWMWloz549uOuuuyDLxqR069YNL7/8MmQDTkXFG1zu+mgV5qw94Kk+4coOGN65kRY7WccBChBcHBBkHVwkuOggokOaILg4JNBBuklwCVJAOz6+chow537Vs/bDgCvesaOnp/gk6ZAlIZdMUISyyGouWfpdGUeUtkEzuLgfPHTokJJlQOtaUG9wmfD9Jry9cJvHhrt6t8SDF1d8gGUoRWPb5lKA4GKueJjVGoKLWSNjPrsILuaLiRktIriYMSomsOnEbuDELkC+1mgINLvQBEaF1oR169YpZzUuWrRIWVVlxuI3uPjrhDe4fLRsN/79hZpZbHCnBnj1ateeGRYqQHDhGNCiAMFFi0qsIwoQXDgOtChAcNGiEus4QQFZVbVv3z707dvXtO6GFVwWbzmC66Yu84hxVuMkzL7jXNOKQ8PCqwDBJbx6W7U3gotVIxd+uwku4dfcij0SXKwYNdrsVAXCCi67jmbjwhcWeLROTYjBiscucqr29LuUAgQXDgktChBctKjEOpxx4RjQqgDBRatSrEcFjFcgrOBSdLIYLf79nY/Xm5+5FDGREcYrQQsMV4DgYngILGEAwcUSYTKFkZxxMUUYTG8EwcX0IQqvgYW5wAstlfNblFfNVsDFT4fXBvZWrgJhBRexosfz87H/RK7HoJ/uvxCt6iQwRFQABBcOAi0KEFy0qMQ6nHHhGNCqAMFFq1IOqXd4E/BmV9XZ5KbAvX86xHnzuxl2cLl6yu/4fftRjzLTbuyC3m1rm18pWhhyBQguIZfYFh0QXGwRxrA4wRmXsMhs+U4ILpYPob4O/P0D8NEwtc3mPYHrv9K3D7YWsAJhB5eHPv0Tn/6x12Pwk5edjht6NA3YAT5oHwUILvaJZSg9IbiEUl17tU1wsVc8Q+UNwSVUylq03eWTge/GqsaffSMw6FWLOmM/s8MOLq/N34KXf/rbo+TN5zXDfwa2s5+y9MhvBQgufkvmyAcILo4Me0BOE1wCks1xDxFcHBfyyh0+vtN1fou8UpoDTXpU/gxrhEWBsIPLF6v34f6P13icu7hdHUy5vnNYnGUn5laA4GLu+JjFOoKLWSJhfjsILuaPkRksJLiYIQq0gQpoUyDs4PLHruO44u2lHutOq5eIufdeoM1a1rK1AgQXW4dXN+cILrpJafuGCC62D7EuDhJcdJGRjVCBsCgQdnD5JyMP5zw7z+OcpEKWlMgsVIDgwjGgRQGCixaVWEcUILhwHGhRgOCiRSXWoQLmUCDs4CJut3lsLvIKT3oUWP34xUiOjzKHIrTCMAUILoZJb6mOCS6WCpehxhJcDJXfMp0TXCwTqtAbmnMcmNjBdX5LchMg9TSgz39C3y970KyAIeDS56WF2HY4y2Pk13edhw4Na2g2mhXtqQDBxZ5x1dsrgoveitq3PYKLfWOrp2cEFz3VtHhb+1cBU3qpTqS2Ae5cbnGn7GW+IeAyavpyLNh82KPkGyPPwsAO9eylLL3xWwGCi9+SOfIBgosjwx6Q0wSXgGRz3EMEF8eFvHyHN8wGPh2l3m91MXDNpxTIRAoYAi6Pf7UeM37b5ZHh//q1xe09W5hIFppihAIEFyNUt16fBBfrxcwoiwkuRilvrX4JLtaKV0it/fVlYN6Tahfn3Ar0fyGkXbJx/xQwBFze+WU7nv3uL4+lI7s2xnND2vtnOWvbTgGCi+1CGhKHCC4hkdWWjRJcbBlW3Z0iuOguqbUbPL5DPcOlZiugcTdr+2Mz6w0Bl+83HMSY9//wSHley1r4YHRXm0lLd/xVgODir2LOrE9wcWbcA/Ga4BKIas57huDivJjTY+sqYAi4bNifjgGvLfao1qRmPBY95LUZyrp60vIgFCC4BCGegx4luDgo2EG6SnAJUkCHPE5wcUig6aYtFDAEXCQVsqREdpeIKsD28QNsISidCFwBgkvg2jnpSYKLk6IdnK8El+D0c8rTBBenRJp+2kEBQ8BFhOvw5I9IzynwaLjk4d5okBRnB03pQ4AKEFwCFM5hjxFcHBbwINwluAQhnoMeJbg4KNgVuZq+D3iru+sMl6QmQN32QM+HKY7JFDAMXAa+/ivW70vzyDHr1u7o1jzFZPLQnHAqQHAJp9rW7YvgYt3Yhdtygku4FbdmfwQXa8ZNd6t3LQGm91ebbXA2cMvPunfDBoNTwDBwuf2DPzB3/UGP9S9c1RFXnd0wOG/4tKUVILhYOnxhM57gEjapLd8RwcXyIQyLAwSXsMhs/k5Wvw98dZdq5xlXAldONb/dDrPQMHB57ru/MOWX7R657+7dCg9c3Nph8tNdbwUILhwPWhQguGhRiXVEAYILx4EWBQguWlRyQJ1V/wMWPg+k73c5e8FYoPejDnDcWi4aBi4f/L4Lj3253qPW5Wc2wMThnaylHq3VVQGCi65y2rYxgottQ6u7YwQX3SW1ZYMEF1uGNTinjm0HouKAxHrBtcOndVfAMHBZ9Pdh3DBtucehs5sk4/Pbe+juIBu0jgIEF+vEykhLCS5Gqm+tvgku1oqXUdYSXIxSnv1SAf8VMAxcth/JQu8XF3osTk2MwYpHL/LfAz5hGwUILrYJZUgdIbiEVF5bNU5wsVU4Q+YMwSVk0rJhKqC7AoaBS9HJYrT493c+Dm1+5lLEREbo7iQbtIYCBBdrxMloKwkuRkfAOv0TXKwTKyMtJbgYqb6J+j603pUGOSbRREbRlNIKGAYuYki35+bjYHqux6Z5D1yIlqkJjJJDFSC4ODTwfrpNcPFTMAdXJ7g4OPh+uE5w8UMsu1Y9vAl4s6vLu7hkoMm5wNUf2tVbS/tlKLhcNek3rNh5zCPg9Bu7oFfb2pYWlMYHrgDBJXDtnPQkwcVJ0Q7OV4JLcPo55WmCi1MiXYGff38PfDRcrdC8F3D9lxTGhAoYCi7/+mQNZq/a55HlqcGn4/ruTU0oE00KhwIEl3CobP0+CC7Wj2G4PCC4hEtpa/dDcLF2/HSxftlkYO5YtamzRwGDJurSNBvRVwFDwWXivC2YOO9vj0e3nN8cjw44TV8P2ZplFCC4WCZUhhpKcDFUfkt1TnCxVLgMM5bgYpj05ul4+WRg8StAxgGXTX2fAs691zz20RKPAoaCy+er9uKBT/70GHPJ6XUx+bqzGR6HKkBwcWjg/XSb4OKnYA6uTnBxcPD9cJ3g4odYTqh6dJtrg34Cty6YMdyGgsvyHccwbPJvHl3a1auO7+4934w60aYwKEBwCYPINuiC4GKDIIbJBYJLmIS2eDcEF4sHkOY7SgFDweVAWi66j5/vEVxSIUtKZBZnKkBwcWbc/fWa4OKvYs6tT3Bxbuz98Zzg4o9arEsFjFXAUHAR15s/8i1OFqsirHn8YiTFRxmrCns3RAGCiyGyW65TgovlQmaYwQQXw6S3VMcEF0uFS39jC3MBWR6W3BSIrqZ/+2xRVwUMB5deLy7EjiNZHqe+ufs8tG9QQ1cn2Zg1FCC4WCNORltJcDE6Atbpn+BinVgZaSnBxUj1TdD3/tXAlJ4uQ+QMl5YXAVe8awLDaEJZChgOLtdPXY5fthz22PbWNWehf/t6jJYDFSC4ODDoAbhMcAlANIc+QnBxaOD9dJvg4qdgdqu+YTbw6SjVq9b9gJEf281L2/hjOLg89uV6fPD7Lo+gD1/aFmMubGEbgemIdgUILtq1cnJNgouTo++f7wQX//Ryam2Ci1MjX+L3ry8D855UReh6G3Dpfx0uinndNxxcJi/ahvFzN3kUuqZrYzw7pL15FaNlIVOA4BIyaW3VMMHFVuEMqTMEl5DKa5vGCS62CWVgjiyZCPz+NpBx0PV8v/FAtzsCa4tPhVwBw8Hlu3UHcMeHqzyOXtAqFTNuPifkjrMD8ylAcDFfTMxoEcHFjFExp00EF3PGxWxWEVzMFhED7eEZLgaKr61rw8Fl3b40DHr9V4+1TWvGY+FDvbRZz1q2UoDgYqtwhswZgkvIpLVdwwQX24U0JA4RXEIiKxulAiFRwHBwSc8pQIcnf/RxbsWjFyE1MSYkDrNR8ypAcDFvbMxkGcHFTNEwty0EF3PHxyzWEVzMEgnaQQUqV8BwcBETOz75I9JyCjzWcp9L5YGzYw2Cix2jqr9PBBf9NbVriwQXu0ZWX78ILvrqaanWfvkvcGQL0PMRIKW5pUx3qrGmAJeJ87Zg4ry/PTGIqAIseqgXGqXEOzUujvSb4OLIsPvtNMHFb8kc+wDBxbGh98txgotfctmncvYxYOIZQH7JWYKnDwUGvATEp9jHRxt64je45Ofn44YbbkB2dja++uqrSiVp1aoVtmzZUmG97PwidB8/32fWZXCn+nj16jMrbZ8V7KMAwcU+sQylJwSXUKprr7YJLvaKZ6i8IbiESlmTtzv3/4Blk1QjE+sB960DqkaZ3HBnm+cXuBQXF+Phhx/G7Nmz0a5dO93ARULw3tKdGPf1Bp9ozH+gJ1qkVnN2hBzkPcHFQcEOwlWCSxDiOexRgovDAh6guwSXAIWz+mNTegH71ay2GDIJ6DjC6l7Z3n6/wGXSpEmYM2cOLrvsMnz77be6gktBUTHOm/AzDqXnekTv3bY2pt3YxfZBoIMuBQguHAlaFCC4aFGJdUQBggvHgRYFCC5aVLJpnS0/AgufB3JPAHetBKpE2NRR+7ilGVy+//57PPHEEwqsLFiwALNmzToFXPbs2XOKMr1798amTeoBkxVJ99Wa/fjXp2t9qnx5Rw+0b1DdPorTk3IVOHbsGJKSkhARwT8cHCblKyDLVQVeEhMTKZOXAlWqVIHMirOoCmRkZCAmJgbR0dGUhQqUq4CAi0BuSgr3Njh2mGQeAhLqaHK/atWqmuqxUmgU0AQua9euxfXXX48ZM2agQ4cOmDlzZpngMmrUqFOs/PXXX/Hbb79ptn74e2ux7UiOp37HBgmYOuJ0zc+zonUVKCoqAv8gWDd+4bJc3pzLGw2OFV/Fo6KiUFCgZmcMVzzM3I/8TZEPQgTqWKhARQrw/w/Hh1YFatWqpbUq64VAAU3gMm7cOCxZsgS9erkOhty4cSPWr1+PYcOG4e67767wk08tm/O9/Vqw6R+Mem+Fj6v/u+kcXNg6NQTus0kzKcClYmaKhnlt4VIx88bGbJZxqZjZImJOe7hUzJxxCYlVu38Hfnoc6Pkw0KJ3SLpgo6FVQBO4LFq0SIEVd1mzZg3kdeONNyozMdWqlb+B3l9wkT6ueHsp/th13NNfm7qJ+OG+C0KrBFs3XAGCi+EhsIQBBBdLhMkURhJcTBEG0xtBcDF9iPQzcOrFwJ5lrvYanA0MmgjU7aBf+2wp5ApoApfSVpS3VKwsawMBl3X70jDo9V99mnvrmrPQv329kAvCDoxTgOBinPZW6pngYqVoGWsrwcVY/a3SO8HFKpEK0s7N3wEzS2UNG7OY4BKkrOF+3JTgIiLcOmMlftx4yKNHo+Q4LBrbG3I4JYs9FSC42DOuentFcNFbUfu2R3Cxb2z19IzgoqeaJm5LlogteVU1sE1/YMRMExtM08pSICBw8UfKQGZcpP3tR7LQ56WF8E6SM+HKDhjeuZE/3bOuhRQguFgoWAaaSnAxUHyLdU1wsVjADDKX4GKQ8EZ0u/NX4NsHgMObAM62GBGBoPs0LbiIZ2M/W4tPVqoplutWj8Xi/+uNqKqcdgk68iZsgOBiwqCY0CSCiwmDYlKTCC4mDYzJzCK4mCwg4TBn6zyg5UXh6Il96KyAqcHlYHouznv+ZxSeVM8meHxQO9x0bjOdZWBzZlCA4GKGKJjfBoKL+WNkFgsJLmaJhLntILiYOz60jgp4K2BqcBFDn/3uL7zzy3aPzTXiovDbI30QH80DgOw2lAkudotoaPwhuIRGVzu2SnCxY1T194ngor+mpmkxPwv43yDg/AeAtgNMYxYNCVwB04NLRm4huo+fj8y8Qo+X/+rbGvf0aRW413zSlAoQXEwZFtMZRXAxXUhMaxDBxbShMZX2mVzXAAAgAElEQVRhBBdThUNfYxY8Byya4GqzybnApROAuu317YOthVUB04OLqPHmgq144YfNHmESYyOxbtwlYRWKnYVeAYJL6DW2Qw8EFztEMTw+EFzCo7PVeyG4WD2C5difvg94/WygIEetcN79wEXjbOqwM9yyBLjkFpzE+RN+xuHMPE9UujeviZm3dnNGlBziJcHFIYEO0k2CS5ACOuhxgouDgh2EqwSXIMQz86Pp+4EZlwFHtrisjE8B7lsHRCeY2WraVokClgAX8WHW8j14ePZaH3cGdqiHN0aexSDbRAGCi00CGWI3CC4hFthGzRNcbBTMELpCcAmhuEY3nXMceH8osH8VcNlrwFk3GG0R+w9SAcuAi/j5wCd/4vNVe31cHn1+Mzw2oF2QMvBxMyhAcDFDFMxvA8HF/DEyi4UEF7NEwtx2EFzMHZ+grZMN+ktfB3o+HHRTbMB4BSwFLkUni3H9tOVYsvWIj3JPDDodo85taryatCAoBQguQcnnmIcJLo4JddCOElyCltARDRBcbBTmb+4D6ncCzr7RRk7RFW8FLAUuYnh2fhGGvLUEmw9m+ERy0nVno9/pdRldCytAcLFw8MJoOsEljGJbvCuCi8UDGCbzCS5hEjrU3fz6CjCvZON970eBC8aGuke2b4AClgMX0Ug26Q987VccSs/1SBZVtQo+GN0NXZulGCAju9RDAYKLHiravw2Ci/1jrJeHBBe9lLR3OwQXG8R33SfA57f4OiLgIgDDYisFLAkuEoGthzNx+RtLfM53SYiJxJd3nYuWqcwYYcVRSnCxYtTCbzPBJfyaW7VHgotVIxdeuwku4dU7JL1tmw/MHAkUlnygHRkL3DgHaNglJN2xUeMUsCy4iGTLdhzDNe/8jsKTxR4F61SPxZx7zkNqQoxxqrLngBQguAQkm+MeIrg4LuQBO0xwCVg6Rz1IcLFJuPf9Abw/BMhLB0Z+ArS62CaO0Q1vBSwNLuLIN3/ux90zV/tEtU3dRHw2pgfkoEoW6yhAcLFOrIy0lOBipPrW6pvgYq14GWUtwcUo5UPQ7z8bAQGYM68LQeNs0gwKWB5cRMSJ87Zg4ry/ffTs0aImPrqFB1SaYZBptYHgolUpZ9cjuDg7/v54T3DxRy3n1iW4WDD2siTso+HAJc8BdU63oAM0OVAFbAEu4vxdH63CnLUHfHQ4rV4ipt14DurViA1UHz4XRgUILmEU28JdEVwsHLwwm05wCbPgFu2O4GKxwKXvAz4cBhxaD0QnANd+DjTmB9UWi2LA5toGXESBa95ddsoZL8nxUZh0XWdmGwt4iITvQYJL+LS2ck8EFytHL7y2E1zCq7dVeyO4WCxyL58GpO/3NXrUd0CTcy3mCM0NRAFbgUtGbiGunLT0lDNeRBgeUhnI8AjvMwSX8Opt1d4ILlaNXPjtJriEX3Mr9khwsVjUNswGPh2lGl23PSDgElPdYo7Q3EAUsBW4iAByQKUsG/t50z+n6DGwQz28PLwToqtGBKIVnwmxAgSXEAtsk+YJLjYJZBjcILiEQWQbdEFwsWAQFz4PLBwPdLoGGPASEBVnQSdociAK2A5c3CK8On8LXvnJd8O+3DujQQ1Mu7ELaicyXXIgAyaUzxBcQqmufdomuNgnlqH2hOASaoXt0T7BxcRxlP0sn48G+r946ib8rfOAlheZ2HiaFgoFbAsuItaivw9jzPt/IKegyEe7lGrRePeGLjircVIoNGWbASpAcAlQOIc9RnBxWMCDcJfgEoR4DnqU4GLSYO9aCswaCeQcB6o3AG77BahWy6TG0qxwKWBrcBERtx3Owqjpy7H7WLaPppERVfD4oNNxffcm4dKa/VSiAMGFQ0SLAgQXLSqxjihAcOE40KIAwUWLSmGuc3gT8GZX304bdgZGzw+zIezObArYHlxE8PTcQtz14Sr8suXwKfrL0rGXh3VE6zqJZouN4+whuDgu5AE5THAJSDZHPkRwcWTY/Xaa4OK3ZOF5YPYtwNpP1L7aXQ5c8S5QNSo8/bMXUyrgCHBxKz/h+014e+G2MgMx4pzGePCSNqhZLdqUgXKCUQQXJ0Q5eB8JLsFr6JQWCC5OiXRwfhJcgtNPt6dlSVhcstpcQQ7wTi/gn7+A8x8A+jyuW1dsyLoKOApcJEzfrz+Ie2etRl7hyVOilhATiXv6tMKoc5shqmoV60bVopYTXCwauDCbTXAJs+AW7o7gYuHghdF0gksYxS6rK9mAP/dhYOdi4J7VvvByfCewfQFwtlf6Y4PNZffGKuA4cBG5N+xPxwOfrMGmgxllqt8oOQ6PDmiHfmfUNTY6Duud4OKwgAfoLsElQOEc+BjBxYFBD8BlgksAoun1yO9vA98/rLbWYTgwdIperbMdGyrgSHCROBYXA5/8sQcvfr8ZhzPzygxtjxY18dTlZ6BlaoINQ28+lwgu5ouJGS0iuJgxKua0ieBizriYzSqCi4ERWTYJmPt/vgZc/xXQvKeBRrFrMyvgWHBxB0UOrHxzwVa8u3h7mcvHqkZUwTVdm+BffVsjKZ4bwkI5mAkuoVTXPm0TXOwTy1B7QnAJtcL2aJ/gYmAcTxYBk84D/tnoMqLpecDlbwFJzPhqYFRM3bXjwcUdnQNpuXjuu7/wzZ/7ywxYjbgo3N+3Na7r1gQCMyz6K0Bw0V9TO7ZIcLFjVEPjE8ElNLrarVWCSxgimpsGrJwKbPgSGDHTdS6Lu+z7A3hvAHDJc0Dnm8JgDLuwsgIEl1LRW73nBP7z5Xqs35dWZlxbpFbDU4PPwLkteQiS3gOf4KK3ovZsj+Biz7iGwiuCSyhUtV+bBJcQx/TPWcA39wKFua6OWl0MXPOpb6e5J4BYHgoe4kjYonmCSxlhlP0vX67ZhwlzN+FgeskvWql6vdvWxrjLTkfjlHhbDAQzOEFwMUMUzG8DwcX8MTKLhQQXs0TC3HYQXEIcn0MbgLd7+HYyZDLQ8eoQd8zm7agAwaWCqErKZDn3ZdKircgtODV9cmREFSV18r0XtYKkUmYJTgGCS3D6OeVpgotTIh28nwSX4DV0QgsEFx2jfGQLsHIa0G+8b6P/GwTs+MX1s6h4oM9/gG536Ngxm3KKAgQXDZE+lJ6L8XM34cvV+8qsnRwfhWeHtEf/9vU0tMYq5SlAcOHY0KIAwUWLSqwjChBcOA60KEBw0aJSJXVkVmXRBOCvr11pW0fMAtpcqj605Sfg0xuAc24Fzr3X96wWHbpnE85RgODiR6zX7UtT9r+s2XOizKea16qGG3o0xVWdGyE+uqofLbOqKEBw4TjQogDBRYtKrENw4RjQqgDBRatSFdT74jZA9rK4S+PuwE3f+z6QfRSIr6lDZ2zCyQoQXAKI/rdrD+DZ7zZi/4my978ItAi83HxeM+6B8UNfgosfYjm4KsHFwcH303XOuPgpmEOrE1wCCHzWYaBaqvrg0a3A62f7NjR6PtCwcwCN8xEqUL4CBJcAR4fsf5GzX95asBVZ+UXlttKzTSpu7NEMF7ZORRVmUa5QbYJLgIPRYY8RXBwW8CDcJbgEIZ6DHiW4+BHsrfOAX14E0vYA9/4JRHjt7/3sZmD9Z0Dd9kCX0UCH4UBUnB+NsyoVqFwBgkvlGlVY40hmPiYv2oaZy3cjM6+w3LpNa8bjjl4tMaxzoyB7tO/jBBf7xlZPzwgueqpp77YILvaOr17eEVw0KjmlJ7B/tVp5yCSg4wj1+th2IPMfoHE3jQ2yGhXwXwGCi/+alflETkERPl6xB1N/3YE9x7LLbVWWkV1yel0MPauhchYMz7JUpSK46DQYbd4MwcXmAdbRPYKLjmLauCmCSznBLb0n5eu7gVUz1Mo1WwB3r7LxyKBrZlSA4BKCqMxdfxDvLN6OVbuOV9h6amIMLu/UAIM71ccZDWqEwBJrNUlwsVa8jLKW4GKU8tbrl+BivZgZYTHBxUv19P2udMYCKK36AoPfVG+e2AVM7KBenz0KuOQZIDrBiLCxT4cqQHAJYeDX7k3DpEXb8P36AzhZXHFHreskYsiZDTD0rAaoUz02hFaZt2mCi3ljYybLCC5mioa5bSG4mDs+ZrGO4FISiX1/AO/0VsMiQDJ2GxDp9Z7km3td15LSuHp9s4SQdjhIAYJLGIK993gO3v99Fz7/Yw9kT0xlpVvzmspMzKBO9VHNQWmVCS6VjQzeFwUILhwHWhUguGhVytn1HAku+VnApm+BDsN8g/9CCyDriPqz0vtYnD1U6L0JFCC4hDEIMuvy+/aj+GLVXshysoo284tZMZER6NuuDoac2RAXtklFpM03xBBcwjgYLdwVwcXCwQuz6QSXMAtu0e4cBS5yQOTqD4G/S85YuXEO0PR8NXI/PAr89obrWmZWLhoHdLvdopGl2XZUwC9wycnJwbFjx1CvXj1ERERo0qNVq1bYsmWLprpOqiTplH/aeAhfrN6LRZsPo7CStWTJ8VEY1LEBhpzVAGc2SrKlVAQXW4ZVd6cILrpLatsGCS62Da2ujjkKXL4YA/w5U9Wv+13AJc+q10f+Bj4fDZx9o2s2hvtXdB1rbCx4BTSDy5gxYzB//nylx5SUFFxxxRUYO3ZspRYQXCqVCCeyC/DVmn3Ka9XuE5U+0DA5Dnf1aoUzGyehTd3ESutbpQLBxSqRMtZOgoux+lupd4KLlaJlnK22ApfCXGD378D2RcCORUC7wa79KO7y1zfAx9eq17VaA3etME589kwF/FRAM7i8+uqr6NevH5o0aYKlS5fitttuw2effYaOHTtW2CXBxb+IyH6Yz1ftxZer92HHkaxKHxaI6d22Nnq1rYMeLWoqy8usWgguVo1ceO0muIRXbyv3RnCxcvTCZ7utwEVmU2RWxV0adgFGz1OvZW/LcyWb6uOSgWYXAMO8UhyHT3b2RAUCUkAzuJRu/fzzz8fIkSNx++0Vr30kuAQUF+Wh1XtO4KvV+/Dlmn3KrExlRaClR8taCsjIWTG1E2Mqe8RU9wkupgqHaY0huJg2NKYzjOBiupCY0iBLgktuGiD7VY7vAno/puoqZ6/8t7mvzg9tBaqlqj/b8AWQ0gyo18mU8aBRVKAiBQICl507d6Jv376YMmUKevXqVaHCBBd9BuCCzYeVWRhJrSz7Y7QUmYHp3qImzmiQhG7NUxAXVVXLY4bVIbgYJr2lOia4WCpchhpLcDFUfst0bilwkRkTWeq17WdV34d3A7FeZ8FJSmNJbewuIz8BWl9imXjQUCqgK7hkZmbi6quvRmJiIj744ANUraq+GZ41a9Ypff3nP//BmjVrGAWdFMgpOIlFW45i4d9H8MvWo0jLKdTccof6iejSNAnnNE3G2Y1rmA5ksrKyEB8fjypVqmj2iRWdp0BRUREKCgoQG+vM847Ki7gkTJE3YCyqArm5uYiKivL5P0V9qEBpBYqLi5GdnY1q1aqZS5yCbERu+xGFLS4BouI8tsVN7oqI9D2e67xBb6Gw7eWe6+jfJyJizzIUtRmAwtYDURxrz4Q+RgXLdOPEKCEM6tevGRfJKnbnnXfiwIED+Oijj5CcnOxj9rvvvnuKGxMmTMDq1asNcs/+3f65N10BmcXbjuGvg5l+OSwgc07TJJzdOEkBmliD98fIP464uDiCi19RdF7lwsJCyIvg4ht7gsupvwsCLpGRkcqLhQqUp4CAi7y/kQ/OzFAit89D5J8fIHLbT4o5uUOmo7DFxSqYLBiH6D/e8VwXth2M3IFvmcF0R9iQkJDgCD/N6qRmcElPT8cdd9yhfCoxderUU6ClPAe5VCx8oT+cmYefN/2DBX/9g8VbDiMrv0hz51UjquCM+jXQrUVNyAGYRiwt41IxzeFydEUuFXN0+P1ynkvF/JLLsZUNXSqWfQwoygcS66r6z7kfWDlNvT7zOmBwydkq8tNdS4Dp/YEmPYDTLnNlDuMp9o4dv05zXBO4CKxI+mP5lPP111+HmzZlmZic6VJRIbgYM6SKThbjz71pyoGXy7YfxYqdx5DtB8jIWZftZW9MGEGG4GLMWLFarwQXq0XMOHsJLsZpb6Weww4uu5YCS14FDq4F0vcDXW4BBryoSrb1J+CDK9Xr+JrA2O2+kmYd9t1wbyXBaSsVCEIBTeBy8OBBSBax0kXOc1m2bBnBJYgAhOtRAZm1+9Lw+7ajWLbjKFbsOObXjIwPyDRLwTnNa6JatL6b/Qku4RoN1u6H4GLt+IXTeoJLONW2bl8hAxeZTTmwGsjLANqpe1Cw81fgvQGqYKltgDuX+wooKYtlI379M10zKufdb12BaTkV0FEBTeASTH+ccQlGvdA9KyCzTkDGMyNzHJl52jf6C8ic0aCGsqyse/OauoAMwSV08bZTywQXO0UztL4QXEKrr11a1x1c0vYA7/YFMg64JKrRELh/gyqXHBL5TB1f+UqnLN70LVCvA1CjkV1kph9UQBcFCC66yGiPRtbsOeEBmeV+zsiIAr3apKJjo2Q0T62GTo2S0DjFv42OBBd7jKNQe0FwCbXC9mmf4GKfWIbSk4DA5ehWYMcvwI7FwP5VwL1/+pr4TG2gME/92dgdQHyKev3uRcDeFa5sYY27A/1fAGq2DKWbbJsK2EIBgostwqi/EyeLgfUlMzIyKyN7ZDJytc/IiEUp1aIVgDmzcRI6NkrCWY2TkRBTfnYfgov+cbRjiwQXO0Y1ND4RXEKjq91arRRcjmwBarXydfvZekBBtvqzu1cBNVt4gUkfYO9K9fraz4CWfdXr7QuAyFgXtLBQASqgWQGCi2apWPHPvSdK9sgcCwhkRMHT6iXi4nZ1ERddFa1qJ6J13UQ0Snblpye4cIxpUYDgokUl1hEFCC4cB1oUKBNcFowHts0HDm1wAcodvwO1T1Obk6xekt3LXa6cCpzhtaH+2weBFe8A1Ru4lnydey8hRUswWIcKVKIAwYVDJCAFZEZm4/40/KbskTmG5TuOIt3PGRl3xzIL07pOAhrXiELHpqloW6862tWrjhpxUQHZxofsrQDBxd7x1dM7goueatqsrfxM4MCfwMF1KN6/BifajkDyaReqTn54FbDlR/X68reATteo1z897soM5i497gYufka9lmxhsgwszve8O5upSHeoQNgVILiEXXJ7dlhcDGw7nAnZJ+N+bTqQjkIhnABLakKMMiPTtq5rZka+tqlTHbFREQG2yMfsoADBxQ5RDI8PBJfw6GzJXmbfAqz9xGN65gVPIKH3v1RXFjwHLJqgXp89Chg0Ub3evhBY/7lrNqVuR6DuGUCUf/s6LakbjaYCBitAcDE4AHbuPr/oJNbvTcPqPSfwZwnQ7D7mtSY4QOdl078KM9XRpk6ikhBADtFksb8CBBf7x1gvDwkueilpwXYObwL+/gE4sBY4+CfQ+Sag2x2qI7+/BXz/iOc6t+0ViL3a69DHzXOBmVe77ku6YpltkeVeLFSAChiqAMHFUPmd17ls8N94IB0yG/PXQfmagc0HM5BTUBSUGFFVq6BFagLa1K2ONu5ZmjqJaFiyfyaoxvmwqRQguJgqHKY2huBi6vAEb5ykHT6xGzi+C0hI9d38vmwyMHes2sfpQ4GrpqvXcgjk9Es914UprRF5zwr1ftYRQDbQN+/Jgx6DjxRboAK6KUBw0U1KNhSoArLMbNexbCzbvBd7M4sVkPnrYAb26DA7494/4wYagRrunwk0UuZ4juBijjhYwQqCixWipMFGOQ8lL9M3s5ecczJrpPqwZOySzF3usmc5MNUri1dyU9+UxXK448T2QL2OKK7THhk1WqN612s1GMMqVIAKGKkAwcVI9dm3jwKls4pl5xdh4373zEw6Nh3MUK6DnZ2RTmslRKNt3epoXSdRmaHpd0ZdJgOwyHgkuFgkUCYwk+BigiBoNUHORfE+x0RSEM8cDhzd5mqhbgdgzGK1tWPbgdfOVK8T6wIPbFavSx/yKGeoyCGQZexDqTQdslYfWI8KUIGQK0BwCbnE7ECrAlrTIe85nuNaalbyEqDZeTQLMnMTbKlXIxYtayeUvBKVr61qJyhn0rCYQwGCizniYAUrCC4mjlLuCeC7sUpWL/yzEaheH/jXX6rBOceBCU3V6+hqwL/3+zpU+pDH/9vpm8VL2pezVZqdD9RuV64YBBcTjxOaRgVKKUBw4ZAwjQJawaUsg/MKTyp7ZzYfyMDfhzKwfn+aMjuTmeffoZnliVE9Lgrnt6qFlqkJiIqMQMOkODSuWQ2SKEBmb1jCpwDBJXxaW70ngovBEZSzTE7scu1DyT4KPLTV16BxNXyvHz3gOyPyfBNAAMddHvwbSKijXk+7BCg+CdTr5MrqJftYYhL9dprg4rdkfIAKGKYAwcUw6dlxaQWCAZfy1DyQlluSCCBD+Spws+NIFoqCSNNcuq/46KoKwMirSc1qaFTyfeOa8WiUHA9JHMCinwIEF/20tHtLBJcQR1j2mRzf6docL4Ay8mPfDsc3AvLS1Z8JuFRLVa9fOR1I26te374EqHOGev2u1+nzKc2BETOB1La6O0Vw0V1SNkgFQqYAwSVk0rJhfxUIBbiUZ8O6fWnKnhmBGfkqszSHM/L8NVlTfVl+pgBNcpwHbJoI1KTEoyaXoGnS0LsSwcVvyRz7AMElyNBnHwMOrAEOrgUOrAP6vwDIXhF3efk0QA5adJd71wDJzdTrKT2B/avV65t/BBp1Va9nDAbkPBQp0u7VH/meLi/7XmSJWGK9IB2p+HGCS0jlZeNUQFcFCC66ysnGglEgnOBSlp2Sqnnr4Uxs+ycTWz2vDMjZMzpO0Ph0LVnP3LM1jWrGo4nM1tSU2ZtqaJAUx9maMgJFcAnmt8xZzxJcKol3QbZrxkSWcsmMiSy18p4ReaMzIJvk3eXmn4BG56jX7w0Adv6qXl/7OdDyIvX6s5uB9V6Zvi5/G+jklQlsy0+upV5127v2uBhUCC4GCc9uqUAAChBcAhCNj4RGAaPBpSKv/jqQgV1Hs5QUzXuOZ7u+HstRvpf9NaEq9ZNi0VSZrXEBjczSyGyNgE1yfFSoujV1uwQXU4fHVMYRXErCIZvf5SDGxl19Z0Sm9wd2LVFjdt0XQIve6rUcwCgHMbrLkMlAx5JDGeVn39wL/PGeer//f4FzblOvd/wCyFkrSU2ApMaulwkLwcWEQaFJVKAcBQguHBqmUcDM4FKRSIcz81wQ4wM1LrjZn5ar634abzsSY9XZGneiANc+m3jUT4pDZIQ999YQXEzzK2t6QxwBLpIuWNkAvwto2MV3j8gPjwK/vaHGaegUoMNw9frre4BV/1OvB7wEdBmtXv/4GLD0dfX6wv8Dev1bvV73KbDlRxcMJTUCGvdwZfGyWCG4WCxgNNfRChBcHB1+czlvVXCpSEVJAiDwUtZMjfwsVPtqhFlkqZlrhsaV/Ux5yXK0mtVQPTbSXMH3wxqCix9iObyqLcBFzitxZ+aSs0zqe51dIlm7VryjRrnfeKDbHep1aXAR6BD4cJdfXwHmjVOv5Vlpw13+mA4sfUM5pBH1OgAt+riWddmsEFxsFlC6Y2sFCC62Dq+1nLMjuFQWAVlmtvtoNvaeyMFuWYqmLEPLwd7j2dh1NFu3dM6l7RBwEYBpmOxK6yx7a2Smpk6NWDRMikdsVERlpht2n+BimPSW69j04CKzJbuXAif2AMd3AK37AWdcoeosUCFw4S69HwUuGKteL34JmP+Uet3jbuDiZ9RrgRqBG3c58zpgsNcMzF/fAHPuB5KbADUaA236+c7IWC7igRlMcAlMNz5FBYxQgOBihOrss0wFnAgulQ2FtJyCktka11I0SRQgUCOws/1IVmWPB3xfkgbUrRELyYhWt3qsAjT1asQpS9BSE6KRmhiDOtVjA24/mAcJLsGo56xnDQeXXUuBjV8CuWmuV9sBgMCDuyx5FfjpcfW62+1Av+fLB4/ONwEDvUDmz5nAF2PU+mdcCVw5Vb2Wze9fjlH3mMj+lbOud9Yg0OAtwUWDSKxCBUyiAMHFJIGgGQDBxb9RUFwMuPbXlCQLKJmtcS9L238iJ2TZ0MTSKlWAlHgXxLhftRJc39dWfhaL1JLrJB0TCRBc/BsnTq4dcnDZuwJYOa0kK9duF5h4g8eqGcDXd6shKD3jsfYTYPYt6v12lwPDvPacbP4OmDlCvS8zMt5npWxfAEjmruSmro3vzS4ABG5Y/FKA4OKXXKxMBQxVgOBiqPzs3FsBgou+46FQ9tecKEkaUJIBzRtyjmTm69thBa3JIZxuqPGATkKsF+S4gEdmeaKrVrxMjeAStrBZviO/wCV9H1CYB8hBh+6y7WfXjIh7xuS0QcDgN9X7G2YDn45Sr9sOBK7+UL3eOg/4wGvpV8u+wLVe6YEllbCkFHYXSTUsKYfd5cjfwPf/BmJruF51Twc632z5uJjNAYKL2SJCe6hA+QoQXDg6TKMAwSW8ocgtOIndxyTFsyutsyxB23nEtRxNACenoCi8BpX0Fh9dVZ3FUWZsYlG7eow6exMbgYTIk2jdsLYh9rFT6yiggEtEAWKKc13wEREFpLZRHdj9G/DF7a79JVLaXAqMmKXel8MR5ZBEd2neC7j+S6/nfwemXaJeNzgbuOVn9VrSEL/VXb2uczpw+1L1WlIFf/+IaylXcmOgdjug6fnWEdgmlhJcbBJIuuEIBQgujgizNZwkuJgrTsoytKMukJGXJAvYdyIHh9JzlWxomXmFhhssZ9koYJMYg1olS9Zqu5erVY9FLWU/Tqxjz7wxPEDhMCDzkGupVmFuyaxEB0+vmeu/R8JnXul/m54H3PitatXelcC7fdRryZ512y/q9eHNwJteBy4K9Ny5XL0v2b4mqv0phyj+6y/1fl4GsOJddcYksS7Q5NxwqMI+/FCA4OKHWKxKBQxWgOBicADYvdd7hMOHUbNmTUREmDejFeOlKlBQVIyDJRBzOMMFM8ors+RryfWRzLyQHtKpJSaVLVVzL2OTpWoxkRx/WibQVkwAACAASURBVDQNS53so8CyKa6uivKBgizg0v+qXR9aD7ztBQJ1zgBuVw9UTN+xGtX/11OtL+eN3LtGvU7bC7xyunotp8Y/tNULPNKB8Y3U65jqwCN7fF3//S1187tk55I6LJZSgOBiqXDRWIcrQHBx+AAwk/uccTFTNPS1JT2nAP+UAho36AjYuL8/mpUX0oQCWrwqc6maVwICdY9ODKra9JBPLTpprvP3DyV7RE64HjnnVvXRrMPAh1ep92OTgHtWq/dlNuXF1up1Yj3ggU1ezx8BXvA68DA+BRhbsuwLQNqRA6jxRltfU8ellbqu4bqOTnBtcvcCH+Xne5arMyayzyQqTrPrrGgNBQgu1ogTraQCogDBhePANAoQXEwTCkMNEYj5JyMPx7LyT5nROZqZj4NpOcqsTmaeMXtwvMWRbGmuLGqyLM2dTc2VaEDSRcsSNkkhLTBk+ZJ9DMjY73sye0EOsOA5dfO6pJq76j3V1dwTwPNN1GsBk4d3qdf5mcBzDdRrgYd/7ysbLNw/LQ88yrgve1xqTO2GKgIcknVL9pIMeNG3/YNrgeoNAYEeFkcqQHBxZNjptEUVILhYNHB2NJvgYseo6u+Td1axPcdzSmZrTl2qpoBPWi7MsFStWnTVknNx4hSgkfNxXId9xiGpWjSS46ORFBcFPdNGa1ZezvoQwJDN6wIicoihu8i+kSk9XXtI8rOAyBjgsX90AwuloXElMx7lgYks1cpLV/scux2Ir6lef3YTcLIQECgSQLlwLBCTqNz3K6uYZsFY0W4KEFzsFlH6Y2cFCC52jq7FfCO4WCxgBpkbSDrkjNzCU/beePbllCxVk1meo5nGLlWTCYsacVEqyFRzAY0CNvECNq6vPteRhUhEljrrISl1vcvcsS7wOL4LkD0dpfdoVAYOpe8/egCIild7KA0W0r73Po8XWgKyJMxdZA+J7CVxl5faAhkH1GtZCiZLwtxFNrdLf4l1gKrRQIPOmpdrEVwM+iW1WLcEF4sFjOY6WgGCi6PDby7nCS7miodZrQkEXLT6Iod6Hs3KL0ky4DWL4048UPJV9uvIvp1wlPMi1iG5SgaqV8lGdWTj7cLLfLrdGDMK8VXyPD8bWfsLxFer7gGch1ZdhOiiTM/9TTesQ/Xk2sp9ZQlbpWDRBsg4qPZ5/3qghteG9Vc7qemEpZZsfpdN8O7y1Z2AZNdyn0Vy3r98l2UdWANUjVHvR1fTTVaCi25S2rohgoutw0vnbKYAwcVmAbWyOwQXK0cvfLaHElz89WLv8RxlKVpZGdXk5zKDcyy7ANE5R0rAIwtrilv6dPNe1AQ0rnJIuV+rSjrOzJ2C40jw1FkZM0b5ubt0zn0bR6Aur1oSfQ8aRBzx3O+R9xr2F9fyXC+Kvh9NIg55rnvlvYQdxeqMxs+xY9Ecez33X2r1PvKSW7lmd+KiMGjplUhM24ziqjEoqt4Qkdd95ntI46r/laQiLlmqJSmHZa+KCQrBxQRBsIAJBBcLBIkmUoESBQguHAqmUYDgYppQmNoQM4GLItSSV4HjO9WlWkMm+S6FeuUMQA4aLCmrrvgVB4pr4Vh2Pk5k52PksiGomafevy7+LazNSUVayYzOj9EPoXWEumH9kvwJ2HxSnfH4JvpRtI9QM2kNzH8W60+qMx5fRv8HnSK2efq/In8c/jipZup6PHIG6lc5inTEI704HlOL+uNAsbqHpGWVfUhDNRwuTvK0ISmjfZatyf6calFIiY9Wlrq578lXOWtHvkqignAXgku4FbdmfwQXa8aNVjtTAYKLM+NuSq8JLqYMi+mM0h1c9q5wgUfOcder881ANXXGAp/fAshyJtm8Lq9bFwK1T1N1kXNE5DwRd5F0unKeiLtMvtD1vLvIAYdy0KG7TL0Y2LNMvb75R6BRV8iytfTcAsS8PxCx+3/33P+h8zvYGNMJJ3IKFPC5eedDaJa3CRmIR1pxPMbm3Yx1xc099QdW/R1VUYQ9xbWx92Qt/INkw2IqS9PK3K/jBTul78usT6BppwkuhoXaUh0TXCwVLhrrcAUILg4fAGZyn+BipmiY15ZTwGXfH67N526waH8lUN0rxe7X9wB7flfv3zAHaNhZdXDaJcBuFQxwy89Ag7PLv18CFp4K/7sM2LFIrX/D10CzC9Xr94cC2+ar19d9AbTorV7PHAFs/k69HjETaNNfvV40wbWpXg43rNHY9aw3WJUK1cliKECjgE2W6+txZXZHQMcFO8rPSu4p19kFyMwrNG3QE2MjPUvXKkxU4E5cEBelzPykpZ1AXFwcYmLCP9tjWjFp2CkKEFw4KKiAdRQguFgnVra3lOBi4xBnHVEPGUxpDsR5feq/f5Ur45UbPE6/3HXehrt8+yCw8xfP/YJhM5GVeiaSkkqWLr03ENi5uHxwmHE5sH2BFzjMBlr0Ua8/uBLY+pN6fe1nQMu+6vVHw4G/v1evr/kUaHWxev3pKGDDbPX6qunA6UPV69m3AHIIo3tz+iXP+oLNkS1AUZ5638CT1+V8nBNZBTiRk4/jbsjJLlCWrUl6aTfkKPdL6uUWnDTtwBXgkaVqyfExSK4mMKMuXasWUxWJMVFIiI1EQkzJy+t7Q1JTm1ZJextGcLF3fOmdvRQguNgrnpb2huBi4vBlH1XBQzJGeYPHmo+AfavU+xf+n++MRmkwuOYzoJUXGPgJDgVXzkBWwwtVcJl1DbBpjire1R8CbQeq15/cAGz8snyw+OxmYP1n6v0r3gHaD/MCj1uBtR+r17KHpeMI9Vqg5tgOFTwanOWbztfEYdXDtPyik8qMjXsGR2Z30uS6ZCbH57pkpkdAKK/QvMDj1iU2KkKBG4GchNgoJMZEoppATmzkKd/Lz+Se1PH9Pgpyjg+LeRUguJg3NrSMCpRWgODCMWEaBQguIQyFN3jIbIb3AX7rPgF2L1fB44IHlT0WnlIaDEovZSoNBsNmAO0Gq8+XBoPSMxKV3Z/tCw6Fg99GZrP+Krh8eQew5sPywUKWiknmK3cZ/AZw5nXq9e9vAbuWAnEpLl3aXQbUP1O9f3QbUJCtgonMnLAErUBOQZELeDzL2EqWs5UsbSsPfgplLZzFipzPUy361Fmd8mZ7ZKYoQQGmSLi+L4Gi2EhIYgQWfRUguOirJ1ujAqFUgOASSnXZtl8KEFwqkCv7mHq6eY2GvlmrZJnSrt/U+3LyedPztYNDKTBA6RmHysDgq7uA1e+XDwZz/gWsnKreHzQROHuUel3Z/e/GAssne+oXXvI8Mk8boYKL3Nu2UAWLDsN896gc2+469d29VIvg4dfvpdkqZ+UXKbM7MmtT5qyOG4ZyCnA0PQfpeUVKXQvyTpnSR0ZU8QGasuEnqszlb+66AkOJsVGIqGK26BpjD8HFGN3ZKxUIRAGCSyCq8ZmQKGBrcPEGD9k4nlBb1VCWMe1cooJH19t892CUBovSS5UqA4fSMw6lwUH2kKx4R7VnwItAl1vU61LggEsnAF3HqPe/fwSQWQt3kT0c3e9Sr396HFg5XQWH8+8HzrhSvb/tZ2C3V1atln0A79Pfj+/wOcAwr0occnJyVHAJyWhko3ZQwDurmBwYquzbyXFBz5GMfM++nSNZeTiWma+kqHZ/ldkguxeZvfGezRGYUZbFefb8CADJdRl7gWQGqWQ2SEDIyoXgYuXo0XanKUBwcVrETeyvacFFlgpl7HcdsndiDyB7GOp1UpX87Q1gk1dWqHPvBVpfot6vDBxKzzjoDQ4//gdY+ppqT9+nALHRXeY9Cfz6snrd5z/A+Q+q1z8/Cyx7WwUPgZJOI9X7sjH+4Dr1ft32vpvrdR5zuqdD1tk+NmceBYJJhyzpqLPyC5Vsa5m5JV/L+D4jrxBZeYXIUOoUICuvCBm5rixt7p9bYT9PsFGTfTxB7QMqSZYg+4rCXQgu4Vac/VGBwBUguASuHZ/UWYGgwEXO33BnpZJUsd7pcCv7RF+WMf3xPyDnhOscj96P+s4o/PAoIHDiLv3GA93uUK9Lg0G/54Fut5d/vzJwuGgccN79vuDwy3/V617/BmQDvLsseh5YMF697vkI0PNh9Xrxi8CvE1WwENu893jI/o79q9X7dU73PRld5zgH2xzBJVgFnfN8MOCip0qyTE1gxgU3FYBQngt43KAk9QV+5Gfu7624x8cfLeXMnrKyvPksifMkSCg/cUL1OO1L4Qgu/kSIdamAsQoQXIzV3zm9e4NFfApQQz35G7IH4dAGZBzei4TIIlSRMzYad1O1kY3Vf3pldeo8Cmh/lXp//lPA4pfU64ueAM77V/lv/Ps8Dpz/gHp/wXOAnJXhLr0fAy54SL1e+Dyw0AsMej0CXFgKDOY/7QUWldy/YKwLjtxlyUTgpyfUa7FdfHCXpa8DYoN7j8Y5o12HJLrLnuWAHKLovm9y8Ah20BNcglXQOc+bBVz0VFxmb8oCGpnt8YBRJbNEymxQfqFyyKmdi3spXHlJEGSGSJbCSeIEmVGvXyv51KxxJcvmJMECCxWgAsYrQHAxPgbmt0D2GBxcr85oyKnfTc9T7ZbD9zZ9q95vdznQ8WovMBgPyKyAu5SeESgNBqXv//IC8PMz6vMCDQIP7lL6vkCHwIe7/PoKMG+cei3LoGQ5VHn3ZRmVzIq4i8y2yKyLu8hsi8y6uMuyycDcseXfXz4FkOVYbrA4+wbfGR05QHH3b+r92u2Ami3NPy4MspDgYpDwFuzWjuCiZxiU2R0NkOOe8XHyUri4KFkKp6bBdmd7c0ORzPDERlVFfHRVxEdFIk6+lrzioiOVlNhyLd+7f65nLNkWFXCKAgQXO0RaTg0/uFYFB9lj4J1VauevwF/fqPfbDgDOvFb1XGYrZNbCXWQZkixHcpfS92W2QmYtynvjX/q+7K+Q5VTuUhoMSt/vcQ9wsdcMhmz8lg3g7iJZsy72ApnS92UplCzXchfJOiUbzN3llPtTgO+8ZlhkNmOg154POdV86RtA1SjX+Ryi32mD1PYyDgJHt6jXUofgEbLfLIJLyKS1XcMEl/CEVJbCZcpSuAqXwZVeIifL4oqUfUGyNM69V6igyObTQF4hERjyhhv1e/l5JOKjBHRK6sSUAE8ZUFTWc5whCs/YZy/hV8BvcJF9CAkJCYiLi9NkbatWrbBli9ebOk1P2axS2l5AZi3cRdLZyiF+7rL7d2DDFypYyMbus29U76/71LVHwb2Ho8vNgOyDcBdZSvSj1wxD6Tf2pWcMZHO1ZH5yl9/fBr73WvpU+n7pGQXJKCWZpcoDA8mKdanXnowV7wLfei3NkoxVsgHdXU65PxoY4LX064/3gG+8NpN3KXVflpJ9/291xkJme7w3nx9YA2yaq/Ynm+u9N89nHAAkRnKoory8zzix2VC0gzsEFztEMTw+EFzCo7OevchSOO/EBqW/F0DyBh6fWSNJpuCVSMHuS+Eq0l2WybmAJhLxMa5ZINeMjxuWSs0KldxX66j1ZSmd98+ZRlvPEc+2/FVAM7js2rULo0ePxs6dO5U+rrrqKjz55JOIioqqsE/LgEvuCRUMYhJ9wWLvSmDLT6qfjboALS9SryWd7aIX1OfPut4XDE5ZSlTJjEDpN/6y1Mh7RuCcW4H+L6j9y+ZyyUzlLqVnDFZOA+Z4bfbufBMw8BW1vrzxl8xX7iJnbEjKXHdZNQP4+m71Wvy77HX1Wg7/k7M+3EU2fsshf+4i98V+91Ip2Z/ivcdk6zxg+RTkIgYxNeqgSvMLgNMuU5/PPAQc+Vu9TqgL1Grl71hnfZsoQHCxSSDD4AbBJQwim7iL7HyZ0VETG7i+d2V+c+8JcmV+K8CRtCwUVon0yhCnLqOTw1JZVAUEijwAVOEMkLosTgEob3AqZzmdJGdgoQIVKaAZXG666SZlpmXChAk4cOAAhg4dqoDL4MFeJ2SX0ZNu4OKebZCv0dV8sx6l73dlRXLXqdUSaNlXtWbTHNfmZvf9DsN93/hXNiPwx3Tgm/vU9kp/4l96RqD0G//VHwBf3ak+L7Mpg15Vr0vfl2Vcg9+sAAxK3f9zFvDFbWp9mXEYoh7Yh7UfA3IWiLuUvr/+M0BOL3cXOcBvqNe5HtsXADIro4BHEtD0XED2sbhL1mHg8Cb1ulptILWN3795QWUV87s3PmBVBQguVo1c+O0muIRfcyv2qCWrmJwD5F4Kp8COe1lcySyP/EwOR83JL4QAk7zke/mZ+3v3z+UcIZbyFageK7NBFUBPtGu2SGaC4qIEotS6vt+7ZprkbKIacRV/yM54WEcBTeCSlpaGzp074+OPP8ZZZ52leCfQIgAzadKkCr1VwGXVL8C+VSo4pDQDWvdTn9vyo2splLu07e97gF3pGYHSn/hXOiPwEfClV3paOYPi8re9wKCS+2s/AWZ7HcjXcQQghwC6S+n77YdBOX3cXdZ/Dnx2k3pd+r6cfP6p10nipw8Frpqu1pdlZJ96LR0rfV/2sMg+ETdYNO7qe8DfP38BMqvhnvGQ/ReSecpkheBisoCY1ByCi0kDY0KzCC4mDIoJTdICLqEwW7LAyWyOC2gKkZNfhNwC1yyRfC8/l+xv7u89AFTyTG7JfdfPXW3I9044NyjQeEhSBXVvkUBNJOTsIJkR8kmo4LV0Ll72F5XsRzqjQQ2kJsYE2j2f00EBTeCydetWXHrppViyZAlq13ad+P3ee+/hiy++wFdffVU5uHz2dMUzArKH4/PRajuVzQjIUqMr3vUCg1IzBnIq95VT1fuylOuTG9Tr04cAV71X/v12g4FhM9T7MmMz6xr1+pT73wKzvA7kk83bV3+k1t+zzJVVyl2anOubDleWQf39vQoWKS0A2WDvLllHgLTdrtkOgQ+b7sEguOjwG+2AJgguDgiyTi4SXHQS0ubNGAUuoZJV9vZ4A5H3DJD6fRGyC1yg4w09KiSVAJNAUp7XLJLDl829MrwThpzZIFShY7saFNAELqtWrcLw4cOxcuVK1KhRQ2l21qxZePPNN7F48WJPN/fc47VPouSnc+fOxeqPnkLCt2M89fJb9EPmAHXGI3r7T0iYoy5lym9xCTIHqDMa0TvmI+EbFWzym12EzEHqjMap9/sgc5AKNlE7FyLxa3VGI79JT2QOVmc05H7Cd3egOKa68spvciFyzlfT31ZN24Xov77w2H8ypTnyWqt7MKrkHEfVjD0ojqmBk9GJKI5L0SA9q5RWoKCgAJGRkajCdCgcHBUoIG8y5CVjhUVVQPQoLCykJF4KiB4RERHKi4UKlKdAcXGx8rtT2Z5dKuhSILfwJHIKTirQIzNE8r28cpWfFSKn0PW9q44A0EnkFLoASa3r+t7VhtQtUp4ze0KFF4e0wZVdeVyBkb8LmsDFPeOydOlSpKamKvaWNePy119/neLLZZddhk1z3kDVmcPUN/7Ne6NoxCee64idv6Dqh0PV+816oWjkp173F6HqJ9cqUCEzDieb9cTJi5/z3K9yYA0i5JyQ2CRXnfqdcLKj1wxIznFUObFTvW/TGQsjB5IefcunowLGBBc91LRvGwK4Musie+5YVAXk90begLGoCmRmZiImJoZvSDkoKlRAfm9kSXxSUhKVMlgBWeamzAIJyJQsl8v2/j6/sASQipRECjkF8ip5xr3kTkDIa+mcgJa0WSR5u4Ms0244C71PqxdkK3w8GAU0gUtZe1zGjRuHgwcPatvjsng2MF8O4CtZ6iQHGMo+FXeRTfNyDom7yHKouh2C8YvPWlABLhWzYNAMMJlLxQwQ3aJdcqmYRQMXZrPttlQszPJZpjs3FLn3E7kBybMUTgEk7+Vz6r4h9z6kRwe0Q9u6iZbx2Y6GagIXcfzGG29E9erVjcsqZkf16ZOPAgQXDggtChBctKjEOqIAwYXjQIsCBBctKrEOFTCHAprBZfv27co5Lnv27FEsl3TITz/9NKKjoyv0RLd0yObQi1aEUAGCSwjFtVHTBBcbBTPErhBcQiywTZonuNgkkHTDEQpoBhe3GrI8TNaWa11fTnBxxDjSxUmCiy4y2r4RgovtQ6ybgwQX3aS0dUMEF1uHl87ZTAG/wcVf/wku/irm3PoEF+fG3h/PCS7+qOXsugQXZ8dfq/cEF61KsR4VMF4BgovxMaAFJQoQXDgUtChAcNGiEuuIAgQXjgMtChBctKjEOlTAHAoQXMwRB1oBgODCYaBFAYKLFpVYh+DCMaBVAYKLVqVYjwoYrwDBxfgY0ALOuHAM+KEAwcUPsRxelTMuDh8AGt0nuGgUitWogAkUILiYIAg0waUAZ1w4ErQoQHDRohLrcMaFY0CrAgQXrUqxHhUwXoGwgIvxbtICKkAFqAAVoAJUgApQASoQnAJbtmwJrgE+HZQCIQeX999/H8XFxbj++uuDMpQP21+BYcOGYdKkSUhJSbG/s/QwYAWWLl2KH374AU8++WTAbfBBZyjw2GOPYeDAgejWrZszHKaXASlw5MgR3HXXXZg1a1ZAz/Mh5yjA97TGx5rgYnwMaEGJAgQXDgUtChBctKjEOqIAwYXjQIsCBBctKrGOKEBwMX4cEFyMjwEtILhwDPihAMHFD7EcXpXg4vABoNF9gotGoViN4GKCMUBwMUEQaIJLAc64cCRoUYDgokUl1uGMC8eAVgUILlqVYj3OuBg/BgguxseAFnDGhWPADwUILn6I5fCqnHFx+ADQ6D7BRaNQrMYZFxOMgZCDiwl8pAlUgApQASpABagAFaACVIAKWFwBgovFA0jzqQAVoAJUgApQASpABaiAExQguDghyvSRClABKkAFqAAVoAJUgApYXAGCi8UDSPOpABWgAlSAClABKkAFqIATFAg5uGRkZKCgoICHCjphNGnw8eTJk/jnn39Qq1YtREZGVvpEWloacnNzUadOnUrrsoK9FMjPz8exY8eU2FepUsVeztEbXRU4fPgwEhISEBcXp7ndzMxMyP8nGV8RERGan2NF6yrg79+UwsJCyMb9unXrWtdpWq6rAnKgelFRkab3L7p2zMY8CoQMXLKysvDAAw9g/vz5SmedOnXCW2+9hdTUVMrvUAUWLFiA+++/HzI2pDz99NO4+uqry1RD3oiMHDkSO3fuVO63bNkSY8aMweDBgx2qnnPcln8Mb7zxBl577TXF6ZSUFEyePFn5G1JZkTF26623YsqUKejVq1dl1Xnf4grs2rULo0eP9vyduOqqq/Dkk08iKiqqXM9kjDz33HOeZ+bMmYM2bdpYXAmaX5EC/v5NEWB55plnMHfuXKXZ6tWr495778XAgQMptMMV+Oqrr/Diiy9i8eLFDlfCOPdDBi7yRuPjjz/GzJkzER8fr/xzadGihfIPg8V5CuTk5KB79+7KH//rrrsOP//8M+68807la6NGjU4R5NChQ5g9ezaGDBmCatWq4b333sPUqVPx22+/+fWpqvOUtr7Hq1atwvDhw5W/HR06dMDEiRPxzTffYNGiRRV+Mr5p0yYFhAWMCS7WHwdaPLjpppuUmZYJEybgwIEDGDp0qAIu5X3AIX9vbrvtNuUldZOTkxEbG8u/KVrEtnAdf/+mfPLJJ8p7FRkv8sHJ559/rnzQtmTJEuX/EYvzFJAPSUaNGoU9e/YoM3AEF+PGQMjARf5x9O/fX/kHIUU+ubjnnnvw999/c9mHcfE2rGf3J+EbNmxAdHS0Ykffvn0ViLn++usrtUv+WPTu3Vt5M9u5c+dK67OCdRX473//i40bNyqwKkUg9rzzzoN80tWuXbsyHZPlh1dccQUefPBBPPHEE3jllVc442LdIaDJcllGKn8L5AOys846S3lGoEUAZtKkSae0IZ+6Dxo0CG3btlU+MWVxjgL+/k15/fXX8cUXXyjvW2JiYiBvWi+66CLI/7GGDRs6Rzh66lHAvWxw3rx5ygoAgotxgyNk4CLLOsaPH49LL71U8U7esF5++eVYuXIlatSoYZzH7NkQBWbNmqXMmPz000+e/mXpV/PmzTF27NhKbfrss8/wyCOPYNmyZdwvVala1q5w3333KZ+EC4C4S6tWrcqdRZHZvGuuuQYXXnihMqMnf3sILtYeA1qs37p1q/L/RT4Fr127tvKIwK684RTILV1kv1TXrl3Rp08fZd9ldnY2evTogVtuuUWZdWGxrwL+/k0RUBk2bJiytP3222/Ht99+q6wcIfDad4xo9UzGwvPPP09w0SpYCOqFBFzkk63WrVv7vNFw/5OR5R7169cPgSts0swKyCcU3333nc8bCvlnIss8ZC1xRUVm6eSfiEzTyhtTFnsrIMt/5FNxb6AVGJGlGvKJuXeRZA8yjqTIkjLZZE1wsff4cHvnXv7j/WGYfEDy5ptvlvmmQmbxZCWALEOUGbz09HRlOZDsW6jsb5AzFLWvl/78TREV5MMQ2aMrcLtt2zYcPHhQ2aMrqwRYnK0AwcX4+IcEXMQtefMgVNqvXz/FS864GB9sIy0IdMZl7969GDFihPJJqaxjr1q1qpFusO8wKCAgIuvKH3/8cU9v5c24uJeRCdi6155Pnz5dWVYoM7zuGd8wmM0uwqyA+8OwpUuXepK+VDTj4gYX71nbTz/9FM8++yxWr17NJcxhjl84u/Pnb4rYJTMra9aswYwZMyAfxMrfFPn/I29a5UNZFucqQHAxPvYhAxfucTE+uGayoKw9LvLm8sYbbyx3j8uWLVtw7bXXKmuLZe26lvTJZvKZtgSmgKxHl43206ZNUxqoaI+LbMSXNxfe5eWXX1Y+WZfZGVk+xmJPBcra4zJu3Djl0/Gy9ri46wusuDPUyZ45AeTNmzczJbI9h4nilT9/U6S+JG6Q/VP//ve/ledlZlcyz8n/Icl2yeJcBQguxsc+ZODinVVMPgm9+eabmVXM+HgbZoFMuXfs2FH5R1BWVjE5T0EgRVLZDhgwQHnjKm885SWflrnPWZB1xvJpPIt9FXAvAZJZOskqJiAiKWvdWcUEaGSvlLzpLKtwqZh9x0ZpFewqswAAArFJREFUz+SDD0lVW1ZWsdJ/U+RZ+T8kn6DL5uujR48qf1tk6bKk32axrwKV/U1Zvny5MvP26quvomnTpgrM/vDDDxDIlayX8v3dd9/Nzfn2HSKVeiZ/N2RvnCRskBk5OepD3pfwA9VKpdO9QsjARQ73kjM7Fi5cqBjdvn17vP322zxIUPcQWqdB+UWXDfnuIp+OyqZqKSdOnECXLl3g/pl8quHeu+DtoXySzg2S1ol5IJbKPwh5AyF7FaTIBx8CK+7MUZL0QzJJyVIOgksgCtvnme3btyup9iXroBT5pFz2QknmwtJ/U+S+1LvrrruUrHVSunXrpoAxzxezz5goy5PK/qa4/zdJ2nXZX3f8+HG89NJLygcmUpo0aQLZJ8NzxOw9TiryTlaASKZcvh8xfgyEDFzcrsn0vJxWy38MxgfbDBbIibOSrlSyALnTIpvBLtpgPgVyc3OVT8Xr1avHZTzmC4+pLJLlYZLoQ15aiiw/lEMqOXurRS371PH3b4o7Ba6c28FCBaiAORQIObiYw01aQQWoABWgAlSAClABKkAFqICVFSC4WDl6tJ0KUAEqQAWoABWgAlSACjhEAYKLQwJNN6kAFaACVIAKUAEqQAWogJUVILhYOXq0nQpQASpABagAFaACVIAKOEQBgotDAk03qQAVoAJUgApQASpABaiAlRUguFg5erSdClABKkAFqAAVoAJUgAo4RAGCi0MCTTepABWgAlSAClABKkAFqICVFSC4WDl6tJ0KUAEqQAWoABWgAlSACjhEgf9vvw6KAAAACAj2by3HjW3AenFcToZWkwABAgQIECBAgEBZwHEpryc7AQIECBAgQIAAgRMBx+VkaDUJECBAgAABAgQIlAUcl/J6shMgQIAAAQIECBA4EXBcToZWkwABAgQIECBAgEBZYF51BOPS7/VzAAAAAElFTkSuQmCC",
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"450\" height=\"250\" style=\"\" viewBox=\"0 0 450 250\"><rect x=\"0\" y=\"0\" width=\"450\" height=\"250\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-a29583\"><g class=\"clips\"><clipPath id=\"clipa29583xyplot\" class=\"plotclip\"><rect width=\"329\" height=\"207\"/></clipPath><clipPath class=\"axesclip\" id=\"clipa29583x\"><rect x=\"26\" y=\"0\" width=\"329\" height=\"250\"/></clipPath><clipPath class=\"axesclip\" id=\"clipa29583y\"><rect x=\"0\" y=\"10\" width=\"450\" height=\"207\"/></clipPath><clipPath class=\"axesclip\" id=\"clipa29583xy\"><rect x=\"26\" y=\"10\" width=\"329\" height=\"207\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(91.8,0)\" d=\"M0,10v207\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(157.6,0)\" d=\"M0,10v207\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(223.4,0)\" d=\"M0,10v207\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(289.2,0)\" d=\"M0,10v207\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,206.68)\" d=\"M26,0h329\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,152.73)\" d=\"M26,0h329\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,98.78)\" d=\"M26,0h329\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,44.84)\" d=\"M26,0h329\" style=\"stroke: rgb(232, 232, 232); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(26,10)\" clip-path=\"url(#clipa29583xyplot)\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace07d55b\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0.33,10.35L4.77,82.46L9.2,100.21L13.64,110.82L18.08,118.42L26.95,129.19L31.39,133.3L44.7,142.84L49.14,145.39L66.88,153.71L71.32,155.44L93.51,162.74L97.94,163.99L129,171.42L133.44,172.34L173.37,179.4L177.81,180.08L235.49,187.66L239.93,188.16L310.92,195.15L315.36,195.53L328.67,196.65\" style=\"vector-effect: none; fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 4px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracebae78e\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0.33,196.65L84.63,188.66L89.07,188.16L146.75,180.74L151.19,180.08L195.56,172.34L200,171.42L231.06,163.99L235.49,162.74L257.68,155.44L262.12,153.71L279.86,145.39L284.3,142.84L297.61,133.3L302.05,129.19L310.92,118.42L315.36,110.82L319.8,100.21L324.23,82.46L328.67,10.35\" style=\"vector-effect: none; fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-dasharray: 4px, 4px; stroke-width: 4px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M25,217.5H355\" style=\"fill: none; stroke-width: 1px; stroke: rgb(36, 36, 36); stroke-opacity: 1;\"/><path class=\"ylines-above crisp\" d=\"M25.5,10V217\" style=\"fill: none; stroke-width: 1px; stroke: rgb(36, 36, 36); stroke-opacity: 1;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(26,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(91.8,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(157.6,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(223.4,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(289.2,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xtick ticks crisp\" d=\"M0,218v5\" transform=\"translate(355,0)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" transform=\"translate(26,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(91.8,0)\">0.2</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(157.6,0)\">0.4</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(223.4,0)\">0.6</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(289.2,0)\">0.8</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"237.4\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(355,0)\">1</text></g></g><g class=\"yaxislayer-above\"><path class=\"ytick ticks crisp\" d=\"M25,0h-5\" transform=\"translate(0,206.68)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ytick ticks crisp\" d=\"M25,0h-5\" transform=\"translate(0,152.73)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ytick ticks crisp\" d=\"M25,0h-5\" transform=\"translate(0,98.78)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ytick ticks crisp\" d=\"M25,0h-5\" transform=\"translate(0,44.84)\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"ytick\"><text text-anchor=\"end\" x=\"17.6\" y=\"4.199999999999999\" transform=\"translate(0,206.68)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"17.6\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,152.73)\">2</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"17.6\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,98.78)\">4</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"17.6\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,44.84)\">6</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-a29583\"><g class=\"clips\"/><clipPath id=\"legenda29583\"><rect width=\"78\" height=\"48\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(361.58,10)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"78\" height=\"48\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legenda29583)\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,14.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre;\">y = 1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(31, 119, 180); stroke-opacity: 1; stroke-width: 4px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"73\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,33.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(36, 36, 36); fill-opacity: 1; white-space: pre;\">y = 0</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(255, 127, 14); stroke-opacity: 1; stroke-dasharray: 4px, 4px; stroke-width: 4px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"73\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\" x=\"0\" y=\"0\"/></g><g class=\"g-gtitle\"/><g class=\"g-xtitle\"/><g class=\"g-ytitle\"/></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = np.linspace(0.001, 0.999, 75)\n",
    "\n",
    "fig = go.Figure()\n",
    "\n",
    "fig.add_trace(\n",
    "    go.Scatter(x=p, y=-1 * np.log(p), name=\"y = 1\", line=dict(width=4))\n",
    ")\n",
    "fig.add_trace(\n",
    "    go.Scatter(\n",
    "        x=p, y=-1 * np.log(1 - p), name=\"y = 0\", line=dict(width=4, dash=\"dot\")\n",
    "    )\n",
    ")\n",
    "\n",
    "fig.update_layout(width=450, height=250, xaxis_range=[0, 1])\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When $y $ is 1 (solid line), the loss is small for $p$ near 1,\n",
    "and when $y$ is 0 (dotted line), the loss is small near 0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If our goal is to fit a constant to the data using log loss, then the average loss is:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "L(p, \\textbf{y}) = &  \\frac{1}{n} \\sum_i  [- y_i  \\log(p)  - (1 - y_i) \\log (1 -p)] \\\\\n",
    " = & -\\frac{n_1}{n} \\log(p)  - \\frac{n_0}{n} \\log (1 -p))\n",
    "\\end{aligned}\n",
    "$$\n",
    "Here $n_0$ and $n_1$ are the number of $y_i$ that are 0 and 1, respectively. \n",
    "We can differentiate with respect to $ p$ to find the minimizer:\n",
    "\n",
    "\n",
    "$$\n",
    "\\frac {\\partial L(p, \\textbf{y})} {\\partial p} =  -\\frac{n_1}{np}  + \\frac{n_0}{n(1-p)}\n",
    "$$\n",
    "\n",
    "Then we set the derivative to 0 and solve for the minimizing value $\\hat{p}$:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "0 &= -\\frac{n_1}{n{\\hat{p}}}  + \\frac{n_0}{n(1-{\\hat{p}})} \\\\\n",
    "0 &= - \\hat{p}(1-\\hat{p}) \\frac{n_1}{\\hat{p}}  + \\hat{p}(1-\\hat{p}) \\frac{n_0}{(1-{\\hat{p}})}\\\\ \n",
    "{n_1} (1-\\hat{p})  & = {n_0} \\hat{p}\\\\\n",
    "\\hat{p}  & = \\frac{n_1}{n}\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "(The final equation results from noting that $n_0 + n_1 = n$.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fit a more complex model based on the logistic function, we can substitute $ \\sigma(\\theta_0 + \\theta_1x) $ for $ p $. And the loss for the logistic model becomes: \n",
    "\n",
    "$$ \n",
    "\\begin{aligned}\n",
    "{\\ell}(\\sigma(\\theta_0 + \\theta_1x), y) & =  ~ y \\ell(\\sigma(\\theta_0 + \\theta_1x), y) + \n",
    "(1-y)\\ell(\\sigma(\\theta_0 + \\theta_1x), 1-y)   \\\\\n",
    " & = y \\log(\\sigma(\\theta_0 + \\theta_1x)) + \n",
    "(1-y)\\log(\\sigma(\\theta_0 + \\theta_1x))\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Averaging the loss over the data, we arrive at:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "L(\\theta_0, \\theta_1,\\textbf{x}, \\textbf{y}) =  \\frac{1}{n} \\sum_i  & - y_i \n",
    " \\log(\\sigma(\\theta_0 + \\theta_1x_i)) \\\\\n",
    " & - (1 - y_i) \\log (1 - \\sigma(\\theta_0 + \\theta_1x_i))\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [],
    "user_expressions": []
   },
   "source": [
    "Unlike with squared loss, there is no closed form solution to this loss function. Instead, we use iterative methods like gradient descent (see {numref}`Chapter %s <ch:gd>`) to minimize the average loss. This is also one of the reasons we don't use squared error loss for logistic models---the average squared error is nonconvex, which makes it hard to optimize. The notion of convexity is covered in greater detail in {numref}`Chapter %s <ch:gd>`, and {numref}`Figure %s <gd-convex>` gives a picture for intuition.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{note} \n",
    "\n",
    "Log loss is also called *logistic loss* and *cross-entropy loss*.\n",
    "Another name for it is the *negative log likelihood*. This name refers to the technique of fitting models using the likelihood that a probability distribution produced our data. We do not go any further into the background of these alternative approaches here. \n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fitting the logistic model (with the log loss) is called *logistic regression*. Logistic regression is an example of a generalized linear model, a linear model with a nonlinear transformation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can fit logistic models with `scikit-learn`. The package designers made the API very similar to fitting linear models by least squares (see {numref}`Chapter %s <ch:linear>`).  First, we import the logistic regression module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we set up the regression problem with outcome `y`, the status of the tree, and covariate `X`, the diameter (which we have log-transformed): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "trees['log_diam'] = np.log(trees['diameter']) \n",
    "X = trees[['log_diam']]\n",
    "y = trees['status_0_1']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we fit the logistic regression and examine the intercept and coefficient for diameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept:           -7.4\n",
      "Diameter coefficient: 3.0\n"
     ]
    }
   ],
   "source": [
    "lr_model = LogisticRegression()\n",
    "lr_model.fit(X, y)\n",
    "\n",
    "[intercept] = lr_model.intercept_\n",
    "[[coef]] = lr_model.coef_\n",
    "print(f'Intercept:           {intercept:.1f}')\n",
    "print(f'Diameter coefficient: {coef:.1f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When making a prediction, the `predict` function returns the predicted (most likely) class, and `predict_proba` returns the predicted probability. For a tree with diameter 6, we expect the prediction to be 0 (meaning `standing`) with a high probability. Let's check:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted probabilities: [0.87 0.13]\n"
     ]
    }
   ],
   "source": [
    "diameter6 = pd.DataFrame({'log_diam': [np.log(6)]})\n",
    "[pred_prof] = lr_model.predict_proba(diameter6)\n",
    "print(f'Predicted probabilities: {pred_prof}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, the model predicts that a tree with a diameter of 6 has a 0.87 probability for the class `standing` and a 0.13 probability for `fallen`.\n",
    "\n",
    "Now that we've fit a model with one feature, we might want to see if including another feature like the strength of the storm can improve the model. To do this, we can fit a multiple logistic regression by adding a feature to `X` and fitting the model again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the logistic regression fits a model to predict probabilities---the model predicts that a tree with diameter 6 has a 0.87 probability of class `standing` and a 0.13 probability of class `fallen`.\n",
    "Since probabilities can be any number between 0 and 1, we need to convert the probabilities back to categories to perform classification. We address this classification problem in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
